Skip to content

ingest/ledgerbackend: Add core configuration support for BUCKETLIST_DB_MEMORY_FOR_CACHING#5632

Merged
tamirms merged 4 commits intostellar:masterfrom
tamirms:bucketlist-cache-flags
Mar 26, 2025
Merged

ingest/ledgerbackend: Add core configuration support for BUCKETLIST_DB_MEMORY_FOR_CACHING#5632
tamirms merged 4 commits intostellar:masterfrom
tamirms:bucketlist-cache-flags

Conversation

@tamirms
Copy link
Copy Markdown
Contributor

@tamirms tamirms commented Mar 18, 2025

PR Checklist

PR Structure

  • This PR has reasonably narrow scope (if not, break it down into smaller PRs).
  • This PR avoids mixing refactoring changes with feature changes (split into two PRs
    otherwise).
  • This PR's title starts with name of package that is most changed in the PR, ex.
    services/friendbot, or all or doc if the changes are broad or impact many
    packages.

Thoroughness

  • This PR adds tests for the most critical parts of the new functionality or fixes.
  • I've updated any docs (developer docs, .md
    files, etc... affected by this change). Take a look in the docs folder for a given service,
    like this one.

Release planning

  • I've reviewed the changes in this PR and if I consider them worthwhile for being mentioned on release notes then I have updated the relevant CHANGELOG.md within the component folder structure. For example, if I changed horizon, then I updated (services/horizon/CHANGELOG.md. I add a new line item describing the change and reference to this PR. If I don't update a CHANGELOG, I acknowledge this PR's change may not be mentioned in future release notes.
  • I've decided if this PR requires a new major/minor version according to
    semver, or if it's mainly a patch change. The PR is targeted at the next
    release branch if it's not a patch change.

What

Fixes https://github.com/stellar/go/issues/5600

Disable BUCKETLIST_DB_MEMORY_FOR_CACHING by default in the captive core library. The BUCKETLIST_DB_MEMORY_FOR_CACHING configuration field was introduced in stellar-core version 22.2.0 so we can only apply the default when running a core binary with that version or higher. If we include BUCKETLIST_DB_MEMORY_FOR_CACHING in a core toml file for an older version of core, stellar-core will crash because it will not recognize the configuration field.

Why

In an upcoming core release, the default value for BUCKETLIST_DB_MEMORY_FOR_CACHING will be increased to potentially 3GB. If we omit the BUCKETLIST_DB_MEMORY_FOR_CACHING field in our captive core toml files then operators will see a sudden increase in memory usage by 3GB due to the BUCKETLIST_DB_MEMORY_FOR_CACHING configuration. This extra memory is useful for validators but it is not a useful optimization for watcher nodes.

Known limitations

[N/A]

@tamirms tamirms force-pushed the bucketlist-cache-flags branch from fc761d3 to a6cce78 Compare March 22, 2025 10:35
@tamirms tamirms marked this pull request as ready for review March 22, 2025 11:47
@tamirms tamirms requested a review from a team March 24, 2025 09:55
Copy link
Copy Markdown
Contributor

@Shaptic Shaptic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hate that we have to version parse, but this is as clean as it gets if we're gonna do that.

@tamirms tamirms merged commit 10ea13d into stellar:master Mar 26, 2025
23 checks passed
@tamirms tamirms deleted the bucketlist-cache-flags branch March 26, 2025 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Configure defaults in captive core library for bucketlist cache configuration

2 participants